The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware platform, which leverages microserver technology to enable users to configure the hardware to meet the requirements of a diverse array of applications. Heterogeneous computing is used to boost performance and energy efficiency. In addition, the full spectrum of hardware accelerators is integrated, providing specialized ASICs as well as FPGAs for reconfigurable computing. The project's contributions span across trusted computing, remote attestation, and secure execution environments, with the ultimate goal of facilitating the design and deployment of robust and efficient AIoT systems. The overall architecture is validated on use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. Ten additional use cases are integrated via an open call, broadening the range of application areas.

Recently, there has been a trend of shifting the execution of deep learning inference tasks toward the edge of the network, closer to the user, to reduce latency and preserve data privacy. At the same time, growing interest is being devoted to the energetic sustainability of machine learning. At the intersection of these trends, we hence find the energetic characterization of machine learning at the edge, which is attracting increasing attention. Unfortunately, calculating the energy consumption of a given neural network during inference is complicated by the heterogeneity of the possible underlying hardware implementation. In this work, we hence aim at profiling the energetic consumption of inference tasks for some modern edge nodes and deriving simple but realistic models. To this end, we performed a large number of experiments to collect the energy consumption of convolutional and fully connected layers on two well-known edge boards by NVIDIA, namely Jetson TX2 and Xavier. From the measurements, we have then distilled a simple, practical model that can provide an estimate of the energy consumption of a certain inference task on the considered boards. We believe that this model can be used in many contexts as, for instance, to guide the search for efficient architectures in Neural Architecture Search, as a heuristic in Neural Network pruning, or to find energy-efficient offloading strategies in a Split computing context, or simply to evaluate the energetic performance of Deep Neural Network architectures.

Nvidia launched the Jetson Xavier NX embedded System-on-Module (SoM) at the end of last year. It is pin-compatible with the Jetson Nano SoM and includes a CPU, a GPU, PMICs, DRAM, and flash storage. However, it was missing an important accessory, its own development kit. Since a SoM is an embedded board with just a row of connector pins, it is hard to use out-of-the-box. A development board connects all the pins on the module to ports like HDMI, Ethernet, and USB. A Jetson module combined with a development board looks similar to a Raspberry Pi or other Single Board Computers (SBC).

Claimed to be the smallest, most powerful artificial intelligence (AI) supercomputer, the Jetson Xavier NX has been introduced by Nvidia. Althought smaller than a credit card, the Jetson Xavier NX is designed for robotic and embedded computing devices at the edge. The Jetson Xavier NX module is built around a new low-power version of the Xavier SoC and delivers up to 21TOPS at 15W and 14TOPS at 10W. These parameters make it suitable for AI workloads suitable for small commercial robots, drones and IoT systems in industry, for example high-resolution sensors, optical inspection as well as network video recorders, portable medical devices and other IoT systems. The Jetson Xavier NX is based on the Nvidia Volta graphics processor unit (GPU) with 384 Nvidia CUDA cores and 48 Tensor cores and two NVDLA.

NVIDIA today introduced Jetson Xavier NX, the world's smallest, most powerful AI supercomputer for robotic and embedded computing devices at the edge. With a compact form factor smaller than the size of a credit card, the energy-efficient Jetson Xavier NX module delivers server-class performance up to 21 TOPS for running modern AI workloads, and consumes as little as 10 watts of power. Jetson Xavier NX opens the door for embedded edge computing devices that demand increased performance but are constrained by size, weight, power budgets or cost. These include small commercial robots, drones, intelligent high-resolution sensors for factory logistics and production lines, optical inspection, network video recorders, portable medical devices and other industrial IoT systems. "AI has become the enabling technology for modern robotics and embedded devices that will transform industries," said Deepu Talla, vice president and general manager of Edge Computing at NVIDIA.

On November 6th, NVIDIA introduced the latest member of the Jetson family - the Jetson Xavier NX. With the size that's smaller than a credit card, this module packs a punch. Earlier this year, NVIDIA launched Jetson Nano, the smallest yet the most powerful GPU-based edge computing device. Jetson Xavier NX, much-advanced edge computing device, has the pin compatibility with Jetson Nano making it possible to port the AIoT applications deployed on the Nano. It also supports all major AI frameworks, including TensorFlow, PyTorch, MXNet, Caffe and others.

Today NVIDIA introduced Jetson Xavier NX, "the world's smallest, most powerful AI supercomputer for robotic and embedded computing devices at the edge." With a compact form factor smaller than the size of a credit card, the energy-efficient Jetson Xavier NX module delivers server-class performance up to 21 TOPS for running modern AI workloads, and consumes as little as 10 watts of power. Jetson Xavier NX opens the door for embedded edge computing devices that demand increased performance but are constrained by size, weight, power budgets or cost. These include small commercial robots, drones, intelligent high-resolution sensors for factory logistics and production lines, optical inspection, network video recorders, portable medical devices and other industrial IoT systems. AI has become the enabling technology for modern robotics and embedded devices that will transform industries," said Deepu Talla, vice president and general manager of Edge Computing at NVIDIA. "Many of these devices, based on small form factors and lower power, were constrained from adding more AI features.